Applying natural language processing to automatically assess student conceptual understanding from textual responses

In this study, we applied natural language processing (NLP) techniques, within an educational environment, to evaluate their usefulness for automated assessment of students’ conceptual understanding from short answer responses. Assessing provides insight into and feedback on understanding, which is often overlooked in grading. Students educators benefit formative assessment, especially online education large cohorts, by providing insights as when required. We selected the ELECTRA-small, RoBERTa-base, XLNet-base ALBERT-base-v2 NLP machine learning models determine free-text validity justification level confidence These two pieces information provide key nature understanding. developed a ensemble using high performance assess with accuracies ranging 91.46% 98.66%. addition, proposed general, non-question-specific confidence-in-response model that can categorise response or low 93.07% 99.46%. With strong these being applicable small data sets, there great opportunity implement techniques own classes. Implications practice policy: Students’ be accurately automatically extracted responses Educators students receive required through without overhead traditional assessment. accurate fewer than 100 student questions.